Automatic selection of visual features and classifiers

In this paper, we propose a dynamic approach to feature and classifier selection. In our approach, based on performance, visual features and classifiers are selected automatically. In earlier work, we presented the Visual Apprentice, in which users can define visual object models via a multiple-level object definition hierarchy (region, perceptual-area, object-part, and object). Visual Object Detectors are learned, using various learning algorithmsas the user provides examples from images or video, visual features are extracted and multiple classifiers are learned for each node of the hierarchy. In this paper, features and classifiers are selected automatically at each node, depending on their performance over the training set provided by the user. Thus, changes in the training data yield dynamic changes in the features and classifiers used. We introduce the concept of Recurrent Visual Semantics and show how it can be used to identify domains in which performancebased learning techniques such as the one presented can be applied. We then show experimental results in detecting Baseball video shots, images that contain handshakes, and images that contain skies. These results demonstrate the importance, feasibility, and usefulness of dynamic feature/classifier selection for classification of visual information, and the performance benefits of using multiple learning algorithms to build classifiers. Based on our experiments, we also discuss some of the issues that arise when applying learning techniques in real-world content-based applications.